AttentiveLearner2: A Multimodal Approach for Improving MOOC Learning on Mobile Devices

We propose AttentiveLearner, a multimodal mobile learning system for MOOCs running on unmodified smartphones. AttentiveLearner uses both the front and back cameras of a smartphone as two complementary and fine-grained feedback channels in real time: the back camera monitors learners’ photoplethysmography (PPG) signals and the front camera tracks their facial expressions during MOOC learning. AttentiveLearner implicitly infers learners’ affective and cognitive states during learning by analyzing learners’ PPG signals and facial expressions. In a 26-participant user study, we found that it is feasible to detect 6 types of emotion during learning via collected PPG signals and facial expressions and these modalities are complement with each other.